fix(skills): align skill catalog names with read_skill resolution#250
Conversation
The system-prompt skill catalog advertised each skill by its internal '## Tool:' heading name (e.g. 'k8s_triage'), but read_skill resolves a skill by its loadable name — the frontmatter 'name' / directory (e.g. 'k8s-incident-triage', which is also what the agent card advertises). When the two differed, the LLM called read_skill with the advertised tool name and got 'skill not found', then abandoned the skill. Changes: - buildSkillCatalog now emits one line per skill keyed by the loadable name (frontmatter name, falling back to directory/file) — the exact string read_skill resolves and the agent card advertises. Each line also carries a 'provides: <tool names>' list so the LLM still sees the skill's capabilities for tool selection, without confusing them for the read_skill argument. Script/binary-backed '## Tool:' entries are excluded from 'provides' because they are registered as first-class callable tools (registerSkillTools) and invoked directly by name. - read_skill resolves via a frontmatter-name index (keyed by both the frontmatter name and the directory/file name, normalized for case and underscore/hyphen drift) in addition to the direct path lookup; a miss now returns the available skill names instead of a bare error. Regression tests: read_skill resolves by frontmatter name when the directory differs, normalized variants, flat layout, not-found lists available skills; catalog advertises the loadable name (not the tool heading) while surfacing tool names under 'provides'; and a catalog<->read_skill contract test asserting every advertised name resolves (the gap that let this ship).
…ference) Skills ship more than SKILL.md — helper scripts (shell, python, javascript), reference folders, and additional markdown all live in the skill's directory and reach the running agent via COPY . . at build time, but were invisible to the LLM because read_skill returned only SKILL.md. read_skill now appends a '## Skill files' listing of every other file in a subdirectory skill's folder, relative to the agent root and annotated with the script language, so the model can read or execute them as the skill's steps describe. Flat single-file skills are unaffected. Also documents why skillEntryHasScript (the catalog 'provides' exclusion) stays aligned to registerSkillTools' .sh-only registration: a py/js tool is not yet a callable tool, so it correctly stays in 'provides' and its script is surfaced by the new file listing. Broadening the extension set must wait for registerSkillTools to register those languages, or the tools would vanish from both the tool list and the catalog.
|
Folded in a follow-up (46b91a3): read_skill now surfaces the skill's directory files. Skills ship more than SKILL.md — helper scripts (shell / python / javascript), reference folders, and extra markdown all live in the skill's directory and reach the running agent ( Flat single-file skills ( On the catalog side, the Tests added: file listing includes py/js/sh scripts + reference files and excludes SKILL.md; flat skills append nothing. |
|
Follow-up tracked in #251 — skill-relative file resolution (read referenced markdown and execute referenced |
…ion (#251) A SKILL.md body routinely references sibling files by a path relative to the skill's own directory ("read reference/runbook.md", "run scripts/check.py"). Those files ship in skills/<skill>/ and reach the agent, but nothing resolved a skill-relative reference: cli_execute and the skill-script executor run with CWD = agent root, and only .sh `## Tool:` entries were runnable at all. So a reasonable skill silently failed at runtime. This adds skill-scoped resolution (issue #251, approach 1): - read_skill gains an optional `file` argument — reads a file relative to the skill's directory (confined; no `..`/absolute escape) instead of SKILL.md. Handles "read reference/runbook.md". - New run_skill_script builtin — executes a script bundled in the skill by path relative to the skill dir, picking the interpreter by extension (.sh->bash, .py->python3, .js->node) and running it WITH THE SKILL DIRECTORY AS CWD so the script's own relative reads resolve. JSON args are passed as the script's first positional argument ($1). Path is confined to the skill dir; the subprocess inherits the same egress proxy + env passthrough as cli_execute / skill scripts. Registered independently of cli_execute (a skill may ship only non-.sh helpers). - Shared helpers builtins.SkillDir + builtins.SafeSkillJoin resolve a skill's loadable name to its directory and confine a relative path, used by both read_skill and run_skill_script. - skillEntryHasScript stays .sh-aligned by design: a .py/.js tool is now runnable via run_skill_script while correctly remaining in the catalog `provides:` list — closing the #250 "falls between not-callable and in-provides" note without a code change (comment updated to explain). Docs: the Skill Designer prompt (forge-ui) and writing-custom-skills.md explain skill-relative references and which languages are runnable. Tests: read (file arg) + execute for shell/python/javascript (each proves CWD=skill dir by reading a sibling file), traversal/absolute-escape rejection, unsupported-extension error, unknown-skill, and the SkillDir/SafeSkillJoin helpers. TypeScript is intentionally unsupported (ship compiled .js).
Problem
An agent with a skill whose loadable name differs from its internal
## Tool:heading could never load that skill. Observed live: the LLM was shownk8s_triagein the skill catalog and calledread_skill{"name":"k8s_triage"}, but the skill's loadable name (frontmattername/ directory, also what the agent card advertises) wask8s-incident-triage.read_skillresolves by filesystem path, so it returned{"error":"skill \"k8s_triage\" not found"}, and the model abandoned the skill and improvised rawkubectl.Root cause: three divergent identifiers for one skill.
name:k8s-incident-triage## Tool:heading (entry.Name)k8s_triageread_skillresolutionskills/<arg>/SKILL.md(+_→-)k8s-incident-triageThe catalog advertised the internal tool heading while
read_skillresolved by the skill's loadable name, and the two never had to agree. Unit tests never crossed the catalog→read_skillboundary, so nothing caught it.Changes
buildSkillCatalog(forge-cli/runtime/runner.go) — now emits one line per skill keyed by the loadable name (frontmattername, falling back to directory/file) — the exact stringread_skillresolves and the agent card advertises. Each line also carries aprovides: <tool names>list so the LLM still sees the skill's capabilities for tool selection, without mistaking them for theread_skillargument. Script/binary-backed## Tool:entries are excluded fromprovidesbecause they are registered as first-class callable tools (registerSkillTools) and invoked directly by name.read_skill(forge-core/tools/builtins/read_skill.go) — resolves via a frontmatter-name index (keyed by both the frontmatternameand the directory/file name, normalized for case and_/-drift) in addition to the direct path lookup. A miss now returns the available skill names (available_skills) so the model can retry instead of giving up. Directory-traversal guard preserved.Why both halves
read_skillwith the resolvable name in the common case (directory == frontmatter name).read_skillindex makes resolution robust when the directory differs from the frontmatter name, and turns a dead-end "not found" into an actionable list.Tests
forge-core/tools/builtins/read_skill_test.go— resolves by frontmatter name when the directory differs; normalized variants (k8s_incident_triage,K8S-Incident-Triage); flatskills/<name>.mdlayout; not-found lists available skills; traversal rejected.forge-cli/runtime/runner_skill_catalog_test.go— catalog keys the line on the loadable name (not the tool heading) while surfacing tool names underprovides; and a catalog↔read_skill contract test asserting every advertised name resolves throughread_skill(the integration gap that let this ship).Full
forge-core/tools/...andforge-cli/runtimesuites green;gofmt -w+golangci-lint runclean on both changed packages.